ON CONNECTIONS OF GENERALIZED ENTROPIES WITH 
SHANNON AND KOLMOGOROV-SINAI ENTROPIES 
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Abstract. We consider the concept of generalized measure-theoretic entropy, 
where instead of the Shannon entropy function we consider an arbitrary concave 
function defined on the unit interval, vanishing in the origin. Under mild as- 
sumptions on this function we show that this isomorphism invariant is linearly 
dependent on the Kolmogorov-Sinai entropy. 

1. Introduction 

Dynamical and measure-theoretic (called also Kolmogorov-Sinai entropy) en- 
tropies are a basic tool for investigating dynamical systems (see e.g. [3 |9]). 
They were extensively studied and successfully applied among others in statis- 
tical physics and quantum information. It appeared to be an exceptionally pow- 
erful tool for exploring nonlinear systems. One of the biggest advantages of the 
Kolmogorov-Sinai entropies lies in the fact that it makes possible to distinguish 
the formally regular systems (those with the measure-theoretic entropy equal to 
zero) from the chaotic ones (with positive entropy, which implies positivity of 
topological entropy [TTj). 

The Kolmogorov-Sinai entropy of a given transformation T acting on a probabil- 
ity space {X, T,, fi) is defined as the supremum over all finite measurable partitions 
V of the dynamical entropy of T with respect to V, denoted by h{T,V). As a 
dynamical counterpart of Shannon entropy, the entropy of transformation T with 
respect to a given partition V is defined as the limit of the sequence (^iJ(P„))^^, 
where 

with 7] being the Shannon function given by r]{x) = — xlogx for a; > with 
77(0) := and Vn is the join partition of partitions T~^V for i = 0, — 1. 
The existence of the limit in the definition of the dynamical entropy follows from 
the subadditivity of 77. The most common interpretation of this quantity is the 
average (over time and the phase space) one-step gain of information about the 
initial state. Taking supremum over all finite partitions we obtain an isomorphism 
invariant which measures the rate of producing randomness (chaos) by the system. 
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Since Shannon's seminal paper |20] many generalizations of the concept of Shan- 
non static entropy were considered, see Arimoto [1], Renyi [T7] and Csiszar's survey 
article |1] . The dynamical and measure-theoretic counterparts were considered by 
few authors. De Paly [U] proposed generalized dynamical entropies based on the 
concept of the relative static entropies. Unfortunately it appeared that, despite 
some special cases [151 [IS] the explicit clculations of this invariant may not be 
possible. Grassberger and Procaccia proposed in [7] a dynamical counetrpart of 
the well-known generalization of Shannon entropy - the Renyi entropy, and its 
measure-theoretic counterpart were considered by Takens and Verbitski. They 
showed that for ergodic transformations with positive measure-theoretic entropy, 
Renyi entropies of a measure-theoretic transformation are either infinite or equal 
to the measure-theoretic entropy [22]. The answer for non-ergodic aperiodic trans- 
formations is different, for Renyi entropies of order a > 1 they are equal to the 
essential infimum of the measure-theoretic entropies of measures forming the de- 
composition of a given measure into ergodic components, while for a < 1 they are 
still infinite [23]. In particular, this means that Renyi entropies of order a < 1 
are metric invariants sensitive to ergodicity. Similar generalization was made by 
Meson and Vericat [T2| [T3] for so called Havrda-Charvat-Tsallis entropy [8] and 
their results were similar to ones obtained by Takens and Verbitski in [22]. 

In our approach is based on Arimoto generalization applied to dynamical case. 
Instead of the Shannon function rj we consider a concave function g: [0,1] i— )■ M 
such that lim g{x) = g{0) = and define the dynamical (yf-entropy of the finite 
partition V as 

h{g, T, V) = \imsup- ^ gi^A)). 

The behaviour of the quotient g{x)/ri{x) as x converges to zero appears to be 
crucial for our considerations. Mainly, defining 

Ci{g) := liminf | and Cs(5') := limsup ^ 



we will prove that 

Ci{g) ■ h{T,V) < h{g,T,V) < Cs{g) ■ h{T,V). 

Moreover for g such that Ci{g) = oo, which fullfills some additional condition (e. 
g. for g{x) = y/x), we can find zero dynamical entropy processes (X, E, /i, T, P) 
with a given positive dynamical (/-entropy. 

Taking the supremum over all partitions we obtain Kolmogorov entropy-like iso- 
morphism invariant, which we will call the measure-theoretic (/-entropy of a trans- 
formation with respect to an invariant measure. One might ask whether this 
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invariant may give any new information about the system. We will prove (Theo- 
rem that for g with Cs{g) < oo, this new invariant is linearly dependent on 
Kolmogorov-Sinai entropy. It means that in fact the Shannon entropy function 
is the most natural one - not only it has all of the properties which the entropy 
function should have [5], but also considering different entropy functions we will 
not obtain essentially different invariant. This result might has the other interpre- 
tation. Ornstein and Weiss showed in [H] that every finitely observable invariant 
for the class of all ergodic processes has to be a continuous function of the entropy. 
It is easy to see that any continuous function of the entropy is finitely observable 
- one simply composes the entropy estimators with the continuous function itself. 
In other words an isomorphism invariant is finitely observable if and only if it is 
a continuous function of the Kolmogorov-Sinai entropy. Therefore our result im- 
plies that the generalized measure-theoretic entropy is in fact finitely observable. 
It should be possible to give a more direct proof of the finite observability of the 
generalized measure-theoretic entropy but the proof cannot be easiei0 than the 
proof that entropy itself is finitely observable, see 

The paper is organized as follows: in the next section we give a formal definition 
of the dynamical (7-entropy and establish its basic properties. The subsequent sec- 
tion is devoted to the construction of a zero dynamical entropy process with a given 
positive (^-entropy. Finally, in the last section, we define a measure-theoretic g- 
entropy of a transformation and show connections between this new invariant and 
the Kolmogorov-Sinai entropy. 

2. Basic facts and definitions 

Let (X, be a Lebesgue space and let g : [0, 1] 1— M be a concave function 
with (yf(0) = lim g{x) = Oo By Qq we will denote the set of all such functions. 

Every g E Qo is subadditive, i. e. g{x + y) < g{x) + g{y) for every x,y E [0, 1], 
and quasihomogenic, i.e. ipg-. (0, 1] — )■ M defined by (Pg{x) := g{x)/x is decreasing 
(see PS])! 

For a given finite partition V we define the (7-entropy of the partition V as 

(1) H{g,r):=J29{f^{A)). 

Aev 

For g = rj the latter is equal to the Shannon entropy of the partition V. For 
two finite partitions V and Q of the space X we define a new partition W Q (join 

^Benjamin Weiss personal communication 

^We might assume only that g(0) = 0, but then the idea of the dynamical g-entropy would 
fail, since if Vn+i 7^ Vn for every n and lim g(x) 7^ 0, then the dynamical g-entropy of the 

partition V would be infinite. Therefore, if g is not well-defined at zero we will assume that 
5(0) lim o(a;). 

"'if g is fixed wc will omit the index, writing just (p. 
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partition of V and Q) consisting of the subsets of the form B (1 C where B E V 
and C G Q. The join partition of more than two partitions is defined similarly. 

2.1. Dynamical (7-entropies. For an automorphism T: X X and a partition 
V = {El, ...,Ek} we put 

T-^V := {T~^Eu...,T-^Ek} 

and 

'P^ = Vy T'^V V ... V T~"+V. 
Now for a given g E Qo and a finite partition V we can define the dynamical 
^(-entropy of the transformation T with respect to V as 

(2) h^{g, T, V) = lim sup -H {g, P„) . 

Alternatively we will call it the (^-entropy of the process (X, E, /i, T, P). If the 
dynamical system (X, S,T, /i) is fixed then we omit T, writing just h{g,V). As 
in the case of Shannon dynamical entropies we are interested in the existence 
of the limit of {^H{g,Vn))^_^- g = obtain the Shannon dynamical 

entropy h{T,V). However, in the general case we can not replace an upper limit 
in ([2]) by the limit, since it might not exist. Existence of the limit in the case 
of the Shannon function follows from the subadditivity of the static Shannon 
entropy. This property has every subderivative function, i.e. function for which 
the inequality g{xy) < xg{y) + yg{x) holds for any x,y G [0, 1], but this is not 
true in general (an appropriate example will be given in Section 2.2). Therefore 
we propose more general classes of functions for which the limit exists. It exists if 
g belongs to one of two following classes: 



liin^ = 0[ or C:=^e^o 



< hm < 00 
x-i>o+ ri[x) 



It is easy to show that if g is subderivative then the limit lim g{x)/ri{x) is finite. 
Moreover we will see that values of dynamical (7-entropies depend on the behaviour 
of g in the neighbourhood of zero. We will prove that if (7 G QqVJQq^., then there is 
a linear dependence between the dynamical (^-entropy and the Shannon dynamical 
entropy of a given partition. First we give the following general result: 

Theorem 2.1. If gi, g2 ^ Qo o,re such that 

r ■ f^i(^) / 
hmmt — -— < 00, 

x^o+ g2[x) 

and V is a finite partition of X with finite dynamical g2-entropy, then 

liminf^-%2,^)<%i,P). 
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// additionally lim sup < oo, then 



/i((7i,P)<limsup^-%2,P). 



Remark 2.1. Whenever [0,1] i— t- M zs a nonnegative concave function satis- 
fying (72(0) = and (72(0) — '^^'^ /lave any pair 0<a<b<ooas limit 
inferior and limit superior 0/(71/(72 in 0, choosing a suitable function gi. The idea 
is as follows: construct gi piecewise linear. To do so define inductively a strictly 
decreasing sequence Xk — > 0, and a decreasing sequence of values yu = gi{xk) 0, 
thus defining intervals '■= [xk+i, x^] where g is affine. The only constraint to get 
a concave function is that the slope of g on each interval has to be smaller than 
yk/xk, and increasing with respect to k; this is not an obstruction to approach any 
limit inferior and limit superior for gi[x) / g2{x) , provided that Xk+i > is choosen 
small enough. 



Proof of Theorem \2.1[ Let (71, (72 G Qq and assume that 



'0 

limsup — -— < 00. 
If the hmit hm H{g2,Vn) is finite, then h{g2,V) = and 

< h{gi,V) < hmsup - ^}f^'^] H{g2,Vn) < hmsup ^^^-hmsup -H{g2,Vn) = 

„->oo nH[g2,Vn) x^0+ g2[X) n^oo U 

so we can assume that it is infinite and that hm v^gi(a;) = 00 is also infinite, since 
H{g, Q) < lim ^g^{x) for any partition Q. 

Fix £ > and 5 > such that for x G (0,5] we have 

limmf — — — e < — -— < hmsup — -— + e. 

x^o+ g2{x) g2[x) ^_,o+ g2[x) 

Then for every index n we have 

\g^< 9MB))<\'G8. 

BeVn, /i(B)><5 

where Gs '■= max (71 (x), Gs = min qi(x), and 

x€lS,l] x&lS,l] 

liminf^-£< Yl 9MB))/ Yl ^72 (Mi?) )< hmsup ^+£. 
Therefore 
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E 9MB)) / E 92iKB)) + E / E 92iKB)) 

. ^ / MS)<5 m(S)><5 / t^{B)<5 

lim mi — -— — e < — 



E 92{m) / E 92{^^{B)) 

ti{B)>S / ti{B)<5 



< iim sup — + e. 

^^0+ 92{x) 



Converging with n to infinity and with e and 6 to zero succesively we obtain 
the assertion. In the case of infinite limit superior of the quotient gi{x)/g2{x) 
we can repeat the above reasoning just omitting an upper bound for considered 
expressions. □ 



Theorem 12.11 immediately implies few corollaries 
Corollary 2.1. If the limit lim exists and is finite, then 

h{g„V)= hm 

Corollary 2.2. IfgeG^U Gf", then 

h{g,V) = C{g)-h{V), 

where C{g) = lim 4^. 

Corollary 2.3. If g e U ^?o'\ then h{g,V) = lim iH{g,Vn). 

n—>-oo 

Moreover using similar arguments we might obtain the answer in the case of 
infinite limit lim gi{x)/g2{x) and positive dynamical (72-entropy: 

x^0+ 

Proposition 2.1. Let (71,52 G Go be such that lim gi{x) / g2{x) = 00 and let a 

x^0+ 

finite partition V has positive g2-entropy. Then h{gi,V) is infinite. 
Let us define 

■■= {9 e Go lini ^ = 00 

[ x^o+ ri[x) 

then we can formulate the following fact: 

Corollary 2.4. If g E G^ and a partition V has positive Shannon dynamical 
entropy, then h{g, V) is infinite. 

2.2. Bernoulli shifts. Let ^ = {1, . . . , /c} be a finite alphabet. Let X = {x = 
{^i}i^-oo • ^ ^"^^ 0" be a left shift 

(T{x)i = Xi+i. 
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For any s < t and block [uq, . . . , Ut^s] with G ^ we define a cylinder 

CKujQ, . . . , Ut^s) = {x e X : Xi = Ui^s for i = s, . . . , t}. 

We consider the Borel a-algebra with respect to the metric, which is given by 
d{x,y) = , where = min{|z| : Xi ^ yi}. One can show that Borel cr-algebra 
is the minimal cr-algebra containing all cylindrical sets. Let p = (pi, . . . ,pk) be a 
probability vector, i.e. Pi > for any i and Spj = 1. We define a measure p = p{p) 
on A by setting p{{i}) = Pi- Then pp is a corresponding product measure on 
X = A^. Thus, the static (^-entropy of a partition P-^ = {[1], [2], . . . , [fc]} is equal 
to 

^Mp 'Pn)= 3 {Co~\(^0, • • • , Wn~l)) = J] ^ (P-o " " -P-n-i) 

By the concavity of the function g we have 

■ 1 



where equality holds only when p = p* = (i,...,^). Before calculating the 
dynamical (7-entropy of the partition V"^ with respect to measure pp* , we give the 
following lemma, which proof will be given later: 

Lemma 2.1. If g & Qq, then 

Ls[g) = hmsup and Ci(5() = limmf 

for any positive integer k > 1. 

Therefore, applying Lemma 12.11 for the partition V'^ we obtain 



(3) V(^'^0=li-«^P^^(i) 



1 f 1^ J ^^(9) 'log^! if Cs((7) < 00; 

00, otherwise. 



Remark 2.2. If we consider lower limit instead of the upper limit we would obtain 

n-)-oo n \k^ J I 00, otherwise. 

Therefore we can not replace an upper limit by the limit in the definition of the 
dynamical g -entropy. 

Proof of Lemma \2. 1[ We will show the equality for the upper limit. Proof of 
the equality for the lower limit is similar. Let {xn)'^=i and (m„)5^]^ be such 
that limsup (7(x„)/?7(x.„) = c and x„ G {k~"^" , k~"^"^^) for every n G N. Then 

n—^oo 

— logx„ > — log/c~™"^^. Every function G ^0 is quasihomogenic, so for every 



8 



FRYDERYK FALNIOWSKI 



positive integer n occurs 



Therefore 



9{xn) g{xn) 1 ^ g{k~ 



and 



r]{xn) Xn -logx„ /c"'"" (m„-l)log/c 

?7 {k~^") rrin — 1 ' 

^(a;) gik-"") 

lim sup — — = lim sup — -. 

^_,0+ V{X) n^oc V{k V 

□ 

3. Zero dynamical entropy processes with 5(-entropy of a given 

VALUE 

As we have seen in the previous section in the case of positive Shannon dynami- 
cal entropy and g G the dynamical (7-entropy is always infinite. The case when 
the Shannon dynamical entropy is equal zero is different. In this section we will 
prove that for g G which fuUfill some additional assumptions, i.e. 

lim liminf > 1, 

A^oo x^o+ g[Xx) 

there exist processes of a given dynamical (^-entropy and zero Shannon dynamical 
entropy. This additional assumption is rather weak, since quasihomogenicity of g 
implies that the limit exists and always has to be greater or equal to one. First 
we will prove some technical lemma which is similar to pi]| Lemma 6.3] for utility 
functions with asymptotic elasticity smaller than one. 

Lemma 3.1. Let g E Qo- For A > 1 define 

UiX) := liminf 

Then U is nondecreasing and for every A > 1 U{X) > 1. // additionally g is 
differentiable, then the following conditions are equivalent: 

(i) U{X) is greater than one for every A > 1, 

(ii) lim sup ^^/^-^ < 1. 

Proof. Since g is quasihomogenic for 1 < s < t we have 

tgjx) ^ sgjx) ^ ^ 
gitx) ~ g{sx) ~ 

for every x G (0, 1] and U{1) = 1. This completes the proof of the first part of the 
theorem. Assume now that g is differentiable. Then it is sufficient to show that the 
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following conditions are equivalent 

{i') 3xo > : Vx G (0,xo) VA > 1 g{Xx) < g{x) 

{n') 3xo > : Vx G (0,Xo) g'{x) < 7-^*^^'' 



X 

for some 7 G (0, 1). The proof will be similar to the proof of [101 Lemma 6.3] 
{i') =^ {it') Fix 7 G (0, 1) Let 



Then 



F{X):=g{\x), G{\):=rg{x). 

G"(l) ^g{x) 

g (x) = < 



[ii') =^ {i') Fix 7 G (0, 1). Functions F and G are different iable, F(l) = G(l) 
and 

F'(l)=x(7'(x)<7^?(x) = G'(l). 

Therefore F{\) < G{X) for A G (1, for some e > 0. To show that F(A) < G(A) 
for every A > 1 let A = inf{A > 1: F{\) = G{\)} and suppose that A < 00. Note 
that we must have -F'(A) > G"(A) which leads to a contradiction, since from {ii') 
we have 

F'(X) = xg'iXx) < ^g(Xx) = ^F(A) = ^G(A) = G'(X). 
AAA 

□ 

The main result of this section is the following theorem: 

Theorem 3.1. For every differentiahle g G Q'^ for which 

i ^^ xg'ix) 

(4) hmsup // < 1, 

and every 7 > there exists a process (X, T,, fi,T,V) , such that 

%,P) = 7. 

We will provide a construction of the process with a given entropy. For 7 = it 
is obvious, we can consider systems with trivial dynamics, i.e. a system consisting 
of a single fixed point with trivial measure. Then we have h{g, V) = for every 
function g & Qq. Suppose now that 7 > 0. Before we will prove the theorem we 
will discuss a well-known construction (see e.g. [6]), which is sometimes called 
the standard example. It will allow us to generate systems where we will be able 
to control the growth of static (7-entropies H{g, Vn) for some partition V, and 
therefore to find a process with a given g-entiopj. The system which we construct 
will be a subshift over two symbols. 

Construction of a system. We define inductively sequence {bn)°^i and the 
family of blocks ( (-Bn,j)j2i ) • {^n)'^=i and (r„)J^^ be given sequences of 

V ' / n=l 
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integers and A := {0, 1}. The sequence (fon)^i is defined inductively by 
bo := 2, and bn+i ■= {bnY" for n > 0. 
(5n,i)i=i ) and ( {B'^ ■) ) ^iven by B^^i = i - 1, for i = 0, 1 

/ n=l V ' I 1 y n=\ 

and as concatenation of Cq blocks i?o,i for 1 < j < Then for > 1 we 
define block B 72+1 2 SI'S cl concatenation of r„ copies of the block B'^ i ior 1 < i < bn, 
and block for 1 < j < 6„+i is a concatenation of e„ (possibly different) blocks 
Then by /i„ and /i^ we will denote length of block Bn^i and ^ respectively. 
Therefore 

n— 1 n— 1 n— 1 n 

= J]^ n J]^ ej and = H H 

i=0 j=0 i=0 j=0 

The basic idea is as follows: we want to control growth of different blocks 
of positive measure. So if the sequence {en)'^=i which gives us the number of 
contatenated different blocks will be constant and equal 2, then the growth of 
blocks will be controlled by concatenating as often as we need as big number of 
copies of one block as we need and the number of copies will be given by values 
of the sequence 

It is easy to see that the system which we will get is the subshift {X, a) consisting 
of sequences for which for every s < t there exist n and i such that [xg, ■ ■ ■ , Xt] is 
a subword of a block Bn,i- The only invariant measure is a Bernoulli measure 
given by assigning to each block Bn^i measure ^{Bn^i) = l/6„. Specifically for a 
given length m G (/in-i;^n) we define measure fi on cylindrical sets C^~^ - we 
find such an integer j G {1, . . . , 2""^ — 1} that the number of different admissible 
blocks is equal to 2^" , now for a given block [uq ■ ■ ■ Wm,-i] we define 



1 there exists such block Bn,i, 

IJ,{C^~^ (wo, . . . , UJ„^-l)) = { that [uo, . . . , ujm-i] is a subword of B^^i 

0, otherwise. 

Such a measure is well defined and invariant. The partition for which we will 
be able to get a given value of the dynamical (^-entropy is the zero-coordinate 
partition V"^ restricted to the subshift. Now if we pass to the subsequence (/in)5^i 
we obtain 



Therefore 



(5) limsup - ^ gifJ'iA)) > limsup ■ bng 
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Moreover any block of length hn is uniquely determined by the block of length 
h'^_i since it is a concatenation of r„_i blocks of length h'^^i- In turn, such a 
block is determined by the block of length hn-i + ctn-i, where by we denote 
length of block which uniquely determines block of length hn-i- Repeating this 
argument n — 1 times we get that any block of length /i„ is determined by a 
subword consisting of its first ^^Iq hi letters. 

Therefore 



limsup— y g (^( A)) >\im sup bng 




We will prove that for functions g G which fuUfill assumptions of Theorem 
13.11 we have 

(6) limsup - ^ (A)) = limsup (-^j j^K ■ 

For this purpose we should check what happens in our construction when 



m e {hn-i, YA=d hi) - Then 



- E ^?(/^(^)) = - E ^(/^(Co""'(^o . . . c^^-i))) 

m ^-^ m ^-^ m 

for some j G {1, . . . , 2"~^ — 1}. According to the definition of measure /i the counter 
of this expression will be piecewise constant - if we denote by {ri-)\_^ a sequence 
of elements of the sequence (rj)^^^ greater than one, the counter will change every 
Tjj, . . . , J or terms. Since g is quasihomogenic the considered functions ip are 
decreasing, therefore the counter of the expression will be increasing with respect 
to j. Denote by 6^"'' the minimum length of blocks (all blocks of the same length), 
for which we get 2^" different cylinders of positive measure in our construction. 
We can focus on the subsequence 

for j = l,...2"-i. Moreover ^]"\ = (l"^""'^-^-^^ j (\}f \ , where 
Qj G {rj^, . . . ,rjj} for j = 1, . . . , 2"~^. It is easy to see that if we want to ob- 

tain equality in (E]), it is enough to show that the sequence (^^j j is non- 
decreasing. According to Lemma 13. 1[ the assumption (jl]) implies that \J (2) = 
liminf 25f(x)/5'(2x) is greater than one. Note that for functions satisfying this 

a:->0+ 
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condition, for sufficiently large n we obtain 



> \_ ./ / 1 + r 



^(2 



n 



3 



\j=0 

n— 1 / /n— 1 



Since f/(2) > 1 there exists an integer A^, such that for n > N we have 
n / n '"i + 1 I > V^(2); which implies that for sufficiently large n, se- 



i=o / \ i=o 



quence ^^j"'' j is increasing. 



Corollary 3.1. If g E is such that U{2) > 1, then 

limsup- ^ = limsup6„5( ( j j^K . 

To complete the proof of Theorem 13.11 it is sufficient to show that for every 
7 > there exists a sequence for which 

In-l 



limsup (-3-) h^hi=^. 



i=0 

We can see that for each positive integer n we have 



where 



^n/ / ^ ' 2ro + ... + 2"ro---r„_i 2ro + . . . + 2"ro ■ ■ ■ r„-i ^ 



Since ^.r^ (^2-2"+' j = 2"+\ the fact that g E implies 

lim a„ = 00. 

n— >oo 

Therefore it is sufficient to show the following lemma: 
Lemma 3.2. For every sequence of real numbers (a„)J^i such that lim a„ = 00 

n— ^00 

and for every 7 > there exist such a sequence of integers (r„)J^^ that 

hmsup = 7. 

2ro + . . . + 2"ro ■ ■ ■ r „_i 
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Proof. Let 



7n := 



2ro + . . . + 2"ro ■ ■ ■ r^-i 
Without loss of generality we may assume that 7 = 1. We will contruct sequence 
{rn)nm inductively due to the index of the next moment at which we concatenate 
multiple copies of a given block. 

Step 1. Since the sequence {an)'^=i converges to infinity there exists an index Nq, 
for which 



(7) 



1 

iVn 



< 



O.N0 



1 - 2-^0 



1 - 2-^0 



< 1 



where [x] is an integer part of x. We may assume that Nq belongs to a subsequence 
('T'fc)fcLi! on which a sequence defined by 6„ := 2"a„/(2" — 1) is increasing. We 
define 



Rt) 



1 - 2-^0 



and 



Ro, for i = 
1, for z = 1,2 



Then by we obtain that 77V0 G ^1 — 1 + and for n = 0, . . . , iVg we have 

7n < iNo- 

Step 2. Let m > 0. Assume that the integers Nq, . . . , Nm-i and r„ for n = 
1, . . . ,Nm-i are already defined. Then there exists such an integer > A^^-i, 
that for every n > N we have 



(8) 1- 



n + 1 



< 



i?0---i?,n-l(l-2^'"--«) 



RQ---R^.,il-2^rn-^-ny 

Moreover the following technical fact, which simple but technical proof we omit 
is true: 

Remark 3.1. There exists such an integer N' G N, that for n > N' we have 



(72 < 



[Rq---R^^,{1-2^^-^~^)\ - i?o---/?™-i(l-2^'"-i 



< 



< 0-1, 



where 



n 



n 



and 



n-lRQ---R^^i{l-2^^-^~-) 



0-2 



(l_2^™-i-")i?i---i?^_i 



n + 1 i?o ■ ■ ■ Rm^i (1 - 2^'-!-"^ 



Now let Nm > max{A^, A^'} be such that A^^ belongs to the subsequence (nfc)^^, 
for which sequence bn^ = / (c2^™-i~"''' + d [l — 2^™-!""'=)) - with c, d > - is 



-a. 
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increasing (existence of such subsequence is guaranteed by the fact that a sequence 
Cn ■= bn/ttn is bouuded and a„ converges to infinity). 
Set 



R. 
and 

r,; := 



_Ro---Rm-i{l-2^^-^- 



-N„ 



Rm, for i = Nrn-l + 1 

1, for i = Nm-l + 2,...,Nm. 
Then by Remark 3, after simple calculations we get that 



/(i?o(l - 2-^")2^°~'^'" + . . . + i?o ■ ■ ■ (1 - 2^—^-)) G f 1 - 1 



and for n = Nm-i + !,•••, we have 7^ < 7Ar„. Eventually 

limsup — = lim 7Ar„ = 1. 

n— >-oo 



□ 



In the proof of Theorem 13.11 the crucial role played the inequality U{2) > 1. 
We used this condition to prove equality ([6]). It is easy to see that we can further 
weaken the assumption - it is sufficient to show that there exists such A > 1 
that f/(A) > 1. Then, applying Lemma [3. II we can repeat the above construction 
defining: 

- an integer k := min{A > 1| U{X) > 1}, 

- an alphabet A := {0,1, . . . , k — 1} and 

- a sequence (e„) as e„ := k for every integer n. 

Corollary 3.2. For every function g G for which 

(9) lim liminf > 1. 

A-!>oo x-i>o+ g[\x) 

and every 7 > there exists a process {X, T,, fi,T,V) , such that 

h{g,V)=^. 

Lemma 13.11 implies that the limit in ^ exists. Moreover this condition is not 
very restrictive - for every g & Qo there is liminf A(7(x)/5'(Ax) > 1. Finally it 
should be noted that there are functions in for which this limit is equal one, 
i. e. it is the case for the function g{x) = x (Inx — 1)^. 

Note that we needed assumption iQ only to show that 

limsup - g{t^{A)) < limsup ■ b^g ( ^ ) . 

i=0 



N 
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To prove this inequality even weaker assumptions are sufficient - condition ([9]) 
implies monotonicity of the sequence (^i""*) when in fact it would be sufficient to 
show just that the upper limit in (jS]) is realized on the blocks of length Yll=o 
Those conditions are usually hard to check and in our opinion will not significantly 
expand our knowledge. Nonetheless we can formulate the following corollary: 

Corollary 3.3. If g G , then for every 7 > there exists a process 
{X,T,, fi,T,V) , such that 

H9,V)>-f. 



4. KOLMOGOROV-SlNAI ENTROPY LIKE INVARIANT 

The basic tool in the ergodic theory is Kolmogorov-Sinai entropy defined as 
a supremum of Shannon dynamical entropies over all finite partitions: 

h^{T)= sup h{T,V). 

V " finite 

It is invariant under metric isomorphism. Following the Kolmogorov proposition 
we take the supremum over all partitions of dynamical (^-entropy of a partition. 
For a given system (X, S, /i, T) we define 

(10) h^{g,T)= sup h{g,T,V) 

V ~ finite 

and call it the measure-theoretic (7-entropy of transformation T with respect to mea- 
sure /i. 

It is easy to see that it is an isomorphism invariant. Ornstein and Weiss [Ti] 
showed the striking result that measure-theoretic entropy is the only finitely ob- 
servable invariant for the class of all ergodic processes. More precisely - every 
finitely observable invariant for a class of all ergodic processes is a continuous 
function of entropy. Of course in the case of G Q^V^Qf^ by Corollary 12. 2l we have 

h,{g,T)= hm 4^-/.,(T). 

We will show that for a wider class of functions, namely for functions for which 

Us ((7) = hmsup — — < 00 

we have 

/i,(^7,T) = Cs(^7)-/i^(T) 

for any ergodic transformation T. This shows that the measure-theoretic (^-entropy 
is in fact finitely observable: one might simply compose the entropy estimators 
[21] with the linear function itself. Our proof will be similar to the proof of [221 
Thm 1.1] where Takens and Verbitski showed that for ergodic transformations 
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supremum over all finite partitions of dynamical Renyi entropies of order a > 1 
are equal to the measure-theoretic entropy of T with respect to measure /i. 

Let us introduce necessary definitions. Let Tj be automorphisms of Lebesgue 
space (Xj,Sj,/ij) for i = 1,2 respectively. Then we say that T2 is a factor of 
transformation Ti, if there exists a homomorphism 0: H- X2 such that 

0Ti = /ii a.e. on Xi. 

Suppose that T2 is a factor of Ti under homomorphism (f). Then for an arbitrary 
finite partition V of X2 we have 

(fe-l \ / k-l \ / fc-l 

g, V T^T \=Hlg,\/ <p-%-'T \ = H lg,\J T^^^'V 
•i=0 / V j=0 / \ i=0 

Hence h{g,T2,V) = h{g,Ti, (()~^V). Therefore 

h^ig,T2)= sup h{g,T2,V)= sup h{g,Ti, (f)~'^V) < h{g,Ti). 

finite 7-"— finite 

This implies the following proposition: 

Proposition 4.1. //T2 is a factor ofTi, then for every function g E Qq 

hi,ig,T2) < h^{g,Ti). 

4.1. Measure-theoretic (^-entropies for Bernoulli automorphisms. An au- 
tomorphism T on (X, is called Bernoulli automorphism if it is isomorphic to 
some Bernoulli shift. The crucial role in the proof of the main theorem of this 
section (Theorem 14. 2p will play a well-known theorem due to Sinai: 

Theorem 4.1 (Sinai, [2T]). Let T be an arbitrary ergodic automorphism of some 
Lebesgue space {X, S, /i). Then each Bernoulli automorphism with h^{Ti) < h^{T) 
is a factor of the automorphism T . 

The following proposition will play a crucial role in our considerations: 

Proposition 4.2. Let T be an arbitrary ergodic automorphism with h^{T) > 
logM for some integer M > 2. Then for every g E Qo 

h,{g,T)>Cs{g)-\ogM. 

Proof. Consider a shift a over all infinite sequences from the alphabet A = 
{0, 1 . . . , M — 1} with the corresponding Bernoulli measure generated by pi = 
■ ■■ = Pm = Jj- It is easy to see that hfj_{a) = logM. From Theorem 14.11 we 
conclude that cr is a factor of T. Therefore applying formula (E]) we obtain 

h^{g,T) > h^{g, a) > h{g,a,V'^) =limsnp-^ (M"") = logM-limsup 

n— >oo n n— >oo ^rj 1,-''^ ) 

Applying Lemma 12.11 completes the proof. □ 
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4.2. Main theorem. Our goal in tliis section is the following result: 

Theorem 4.2. Let T be an ergodic automorphism of Lebesgue space (X, S,/i), 
and g E Qo be such that Cs{g) G (0, oo) Then 



h^,{g,T) 



Cs{g)-h^{T), ifh^{T)< oo, 
oo, otherwise. 



If g G Qq, then h^{g,T) = 0. If g G Qo is such that Cs{g) = oo and T has positive 
measure-theoretic entropy, then h^{g,T) = oo. 

We need a few preliminary lemmas. 

Lemma 4.1. If T is an automorphism of the Lebesgue space (X, then for 

every g E Go 

h,{g,T"')<mh,{g,T). 
Proof. Let V he a. finite partition. Then 

(m-l \ / fc-l /m-l N 

g,T'^\\/ T'^V] = ]imsup-Hlg,\/T-^^n\/T-^V 
i=0 / \ j=0 \i=0 

= m lim sup -^H {g, Vkm-i) < m lim sup -H {g, Vn) 
= mh{g,T,V) 

Taking supremum over all finite partitions we obtain the assertion. □ 
Next lemma will be just a weaker version of Theorem 14.21 

Lemma 4.2. // an automorphism of a Lebesgue space {X, S, fi) is ergodic for 
every m eN, then for every function g E Qo, such that Cs{g) < oo holds 

h^{g,T) = Cs{g)-h^{T). 

If g E Qq, then h^{g,T) = 0. If g E Qq is such that Cs{g) = oo and T has positive 
Kolmogorov- Sinai entropy, then h^{g,T) = oo. 

Proof. Case of g E Qq follows from Corollary 12.21 Suppose that there exists such 
g E Go\Qo which fuUfills assumptions of lemma and for which we have 

Cs{g)-h^{T)-h^{g,T)>0. 

Then applying Lemma 14.11 to the transformation T"^ and using equality 
/i^(T™) = mh^{T) (see P Thm 4.3.16]) we obtain 

Cs((7)/i^(T™) - h^{g,T^) >m{Cs{g)h^{T) - h^{g,T)) ^ oo as m ^ oo. 

Therefore for sufficiently large m there exists an integer M for which 

(11) h^{g,T^) < mh,{g,T) < Cs{g)\ogM <mCs{g)h^{T) = Cs{g)h^{T^). 



18 



FRYDERYK FALNIOWSKI 



Proposition 14.21 applied to the transformation guarantees that for every g E Qo 
with positive (finite) Cs{g) we have 

(12) h^ig,T"^)>Cs{g)\ogM. 

Comparing ffTTjl and f|T2|) we obtain the contradiction, which imphes that 

h,{g,T) = Cs{g)h,{T). 

If Cs{g) = oo and h^{T) > then there exists such integer m > that 

h^{T"')=mh^{T)> log M 
and by Proposition 14.21 and Lemma 14.11 

h,ig,T) = h,ig,T"^) = oo 
which completes the proof. □ 

Proof of Theorem \4-^ If h^{T) = theorem is true, since for any partition V we 
have 

0<hig,V)<Csig)hiV) = 0. 

Suppose that < h^{T) < oo. Automorphism T is ergodic. Therefore it has 
factor which is a Bernoulli automorphism T' with entropy h^{T) = h^{T'). Every 
Bernoulli automorphism is mixing, so T"* is ergodic for each m. Applying Lemma 
14.21 we obtain 

K{gX) = Cs{g)h,{T') = Cs{g)h^{T). 
Since T' is a factor of T, so Proposition 14.11 implies that 

Cs{g)h^{T) = Cs{g)h^{r) = h^{gX) < K{g,T) < Cs{g)h,{T) 

which completes the proof of the case of finite h^{T). If h^{T) = oo, then Propo- 
sition 14.21 implies that 

h,{g,T)>Cs{g)\ogM 
for every M > and the theorem is proved. □ 

4.3. Case of g E Q^. We will prove that for every g G and any aperiodic 
automorphism T the measure-theoretic (^-entropy of the transformation T with 
respect to /i is infinite. Since we omit the assumption of ergodicity we will use 
different techniques mainly based on the well-known Rokhlin Lemma which guar- 
antees existence of so called Rokhlin towers of a given height covering sufficiently 
large part of X. Using such towers we will find lower estimations of gf-entropy of a 
process similar to one obtained by Frank Blume in |2], [3], where he proposed, for 
a given sequence (an)5^i converging to infinity slower than n, a construction of 
a partition into two sets V, for which lim H{Vn)/0'n = oo. We will assume that 
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we have an aperiodic system, i.e. system (X, S,/i, T) for which 

/i ({x G X : 3n G N T"x = x}) = 0. 

If Mq, . . . , Mn-i C X are pairwise disjoint sets of equal measure, then r = 
(Mo, Ml, ... , Mn-i) is called a tower. If additionally M C X and M^ = T''M for 
k = 1, ... ,n — 1, then r is called Rokhlin tower0 By the same bold letter r we 
will denote the set [JI~qT^M. Obviously fi^r) = nfi{M). Integer n is called the 
height of tower r. Moreover for i < j we define a subtower 

j 

r/ := {TM, T^M) and r/ = |J T''M. 

k=i 

In aperiodic systems there exist Rokhlin towers of a given length and covering 
sufficiently large part of X: 

Lemma 4.3 (Rokhlin, [IB])- aperiodic transformation of Lebesgue space 

{X, T,, fi), then for every e > and every integer n >2 there exists a Rokhlin tower 
T of height n with fJ^i^T) > 1 — e. 

We now give a definition of an independent collection of sets relative to a Rokhlin 
tower. We will associate to such collections certain partitions, which will be anal- 
ogous to Bernoulli partitions. Let r = (M, TM, . . . , T"~^M) be a Rokhlin tower. 
We say that / G E is independent in r, if 

{T-^I n T'^M), M\T~^{I n T^M)]l~J^ 

is a collection of pairwise disjoint partitions of M. In other words for any 
j G {0, . . . , n — 1} we have 

, {T-' (I n T'M) n T- (/ n r-M)) = r^Tm^^ nT'M) 

We can assume that 

fi{I n T'^M) = for k = 0,...,n-l. 

If T is aperiodic, then such collection exists in every tower since every aperi- 
odic system has no atoms and for any nonatomic Lebesgue space {X, T,,fi), every 
measurable set A and each a G [0, fi{A)] one can find a set S C A of measure a. 

In order to show that the measure-theoretic (^-entropy is infinite we need a lower 
bound for the dynamical (^-entropy of a given partition. For this purpose we will 
use Rokhlin towers and we will calculate dynamical (?-entropy with respect to 
a given Rokhlin tower. This leads us to the following definition: Let P be a finite 
partition of X and F G S, then we define the (static) ^f-entropy of V restricted to 



it is also known as Rokhlin-Halmos or Rokhlin-Kakutani tower. 
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F as 

The following lemma gives us estimation for H{g,V) from below by the value 
of (?-entropy restricted to a subset of X. 

Lemma 4.4. Let g E Qo- Let V be a finite partition such that there exists a set 
E eV with < fi{E) < 1. IfF G S, then 

H{g,V) > HF{g,V) - \g'_ (1/2) | - 

where rf^ax := max \g{x) - g{y)\. 

Proof. By the mean value theorem we have 

g{fi{A)) - g{f,{A n F)) = g'^{x^) {^^{A) - ^^{A n F)) , 

for any set of measure smaller or equal to 1/2, where Xq G {ij^{A fl F),/x(y4)). 
Concavity of g implies 

E {9{^^{A))-g{^,{AnF)))>g'^{l|2) KA\F) > -\g'M/2)\. 

Ai(^)<l/2 fj.{A)<l/2 

Eventually 

H{g,V)-HF{g,V) + d^^^ > -\g'_{l/2)\ 
which completes the proof. □ 

Now we give an estimation from below for the (^-entropy restricted to a given 
Rokhlin tower. First, by we will denote a partition into two sets {I,X\I}, 
wfor a measurable set /. Then the following lemma is true. 

Lemma 4.5. Let r = (M, TM, . . . , T^^-^M) be Rokhlin tower of height 2n, / G S 
be an independent set m r. If g E then 



Proof. Independence of / in r implies that the partition 

is a partition of Tq~^ into 2" sets of equal measure 2~'^ ii{tq~^) . Therefore 



2"+i 
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□ 

We need to show the continuity of H{g,Vn) with respect to the partition Vn if 
n is fixed. First in the space of all partitions of X into m sets, which will be denoted 
by Q3m, we consider pseudometric, which once factored to classes of partitions 
modulo measure zero becomes a metric. For any V = {Ai,i G {l,...,m}}, 
Q = {Bi, i G {1, . . . , m}}@ we define 

m 

d(V, Q) = min V/i(A,Afi,(,)) 

TV ' • 

i=l 

where the minimum runs through all permutations of the set {1, . . . iVti\ and by 
A we denote a symmetric difference of two sets. Then: 

Lemma 4.6. If g E Qo is a continuous function, then H{g, ■) is uniformly contin- 
uous on (53m, d)- 

Proof. Fix 6 and V, V such that d{V, V) = 6. Then (after ordering partition V' 
in such way that the distance is realised) 

m m 

6 = Y^f^iAAB,) > HA) - KB,)\ . 
1=1 1=1 

Let Am := {(pi, . . . ,Pm) : Pi > and Yl^iPi = 1} be a m-dimensional simplex. 
Continuity of a transformation 

m 
1=1 

implies that H{g,-) is continuous on the compact set and therefore it is uniformly 
continuous on 

□ 

Theorem 4.3. Let g G and T be an aperiodic automorphism of a Lebesgue 
space {X, S, /i) . Then 

h,,{g,T) = oo. 

Proof. We will prove that for any M > there exists a partition = {E, X\E} 
such that h{g,V) > M. We define recursively a sequence of sets En G S. Let 

Eo:=0, iVo:=5o:=l. 

Let 77, > and assume that we have already defined En-i, iVn-i and 6n~i- Using 
Lemma [4.61 we can choose 5„ > such that 

(13) 5n < ^6n-l 

^If fc < TO we can treat any partition of X into k sets as a partition of X into to sets, since it 
becomes a partition into to sets by adding m — k copies of an empty set. 
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(14) 



< 1 



for any F G S, for which /i(i?.„_iAF) < 25„. 
Since 

hm — — = oo, 

x^o+ rj[x) 

we can choose such A'^ G N that 

By Lemma US] there exists M„ G S, such that = (M„, TM„, . . . , T^^^-^M^) 
is a Rokhhn tower of measure fi{Tn) = Sn- Let /„ C t„ be an independent set in 
r„ and 

En := (-E„_i\r„) U In- 

Then 

^{En-lAEn) < /i(T„) = (5„,. 

for all positive integers n. By f[T^ we have 5^ < 2~" and we conclude that 
{1e„)'^=o is a Cauchy sequence in Li{X). Therefore there exist _E G S such that 
1e„ converges to l^;. For this set we have 

oo oo 

fi{EnAE) < f^{EkAEk_^) < J2 6k< 26n+i. 

k=n+l k=n+l 

Since En. DTn = In, applying ffUl) and Lemmas 14.41 and H75] we obtain 



H{g,Vl) > H{g,V^:)-l 

> H^,^^.,.-^{g,V^:)-\g'{l/2)\-d, 



> 



> 



/i(r„) log 2^^ ^ /i(Tn) log /i(r, 



2 ■ 2 

-|(^'(l/2)|-rf,,ax-l 

log2 ^ (/i(T„)2-^"-^) 

— - ■ fl{Tn) ■ [Nn + 1 ■ ' , .^„jv 

2 iPn{lJ.{Tn)2 ^"-^ 



From ( 1T5|) we obtain that 



lim 



. log 2 



Nn 



> lim 

n— >oo 2 



> — hm /i(r„) 



^-|(?' (1/2)1 
iVn + 1 (/i(r„)2"^"^^) (1/2)1 +d„ 

Nn ■ <^^(/i(T„)2-^"-l) 
iVn + 1 (/i(T„)2-^"-l) 
iVn ■ (/i(T„)2-^"-l) 



Nn 



> M ■ lim = M. 

n->-oo 
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Since M can be arbitrarily large it completes the proof. 

□ 

4.4. Generator theorem counterpart. In the case oi g E there is no coun- 
terpart of a Kolmogorov- Sinai generator theorem, which says that the measure- 
theoretic entropy of the transformation T is realised on every generator of the 
(T-algebra S. Let us consider Sturm shifts - shifts which model translations of the 
circle T = [0,1). Let /3 G [0,1) and consider the translation 0^: [0,1) t— )■ [0,1) 
defined by (pis^x) = x + (3 { mod 1). Let V denote the partition of [0, 1) given 
by = {[0,/3), [/3, 1)}- Then we associate a binary sequence to each t G [0,1) 
according to its itinerary relative to P; that is we associate to t G [0, 1) the bi- 
infinite sequence x defined by = if 0^(t) G [0, (3) and Xi = 1 if 0^(t) G [/3, 1). 
The set of such sequences is not necessary closed, but it is shift- invariant and so 
its closure is a shift space called Sturmian shift. If /3 is irrational, then Sturmian 
shift is minimal, i.e. there is no proper subshift. Moreover for a minimal Sturmian 
shift, the number of n-blocks which occur in an infinite shift space is exactly n + 1. 
Therefore for zero-coordinate partition P"^, which is a finite generator of a-algebra 
E and for any function g E have 

where ixs is the unique invariant measure for Sturm shift. Thus, 

Kg,V^) < limsup "^g = 0. 

n^oo n \n+lj 

On the other hand since it is strictly ergodic (and thus aperiodic) Theorem 14.31 
implies that for any function g G Q'^ 

h,,ig,T) = cx), 

therefore we have a finite generator, for which the supremum is not attained. 
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